Exercise: texture¶
Local binary pattern (LBP) is a feature used for texture representation and classification. The LBP operator describes the surroundings of a pixel by generating a bit-code from the binary derivatives of a pixel. The operator is usually applied to greyscale images and the derivative of the intensities. Note how a pixel with a value > C is assigned ‘1’ and a pixel with a value <= C is assigned ‘0’.
#import modules and images
import numpy as np
from matplotlib import pyplot as plt
from PIL import Image
import requests
from io import BytesIO
#generate random image from lorenpicsom
#image credit: https://picsum.photos/
url = 'https://picsum.photos/224/224'
response = requests.get(url)
img = Image.open(BytesIO(response.content))
#or you can use image of mine.
img = Image.open('assets/test_image_beach.jpg')
(h,w) = img.size
img = img.resize((h//3,w//3), Image.ANTIALIAS)
img=np.asarray(img.convert("L"))
plt.imshow(img, cmap='gray', vmin=0, vmax=255)#'gist_gray')
img.shape
def lbp(img, posX, posY):
# sample matrix
# [[0 1 2]
# [3 4 5]
# [6 7 8]]
patch = img[posY-1:posY+2,posX-1:posX+2].astype(int)
# subtract values by center pixel
# [[-4 -3 -2]
# [-1 0 1]
# [ 2 3 4]]
patch = patch - patch[1,1]
# read code in circular manner
# [-4, -3, -2, 1, 4, 3, 2, -1]
patch = patch.ravel()
code = np.array([patch[0],patch[1],patch[2], patch[5], patch[8], patch[7], patch[6], patch[3]])
# clip negative to 0 and positive to 1 and reshape:
#[0 0 0 1 1 1 1 0]
code = code.clip(min=0, max=1)
#convert into int from bin
#30
code = code.dot(2**np.arange(code.size)[::-1])
return code
lbp_image = np.zeros(img.shape, dtype=int)
for x in range(1,img.shape[1]-1):
for y in range(1,img.shape[0]-1):
lbp_image[y,x] = lbp(img, x, y)
def plot(imgs, titles=[]):
fig, axs = plt.subplots(1, len(imgs), figsize=(20, 20))
i=0
for ax, theta in zip(axs, imgs):
ax.imshow(np.clip(theta,0,255), cmap='gray')
if titles: ax.title.set_text(titles[i])
i=i+1
plt.show()
plot([img,lbp_image])
import plotly.graph_objects as go
from plotly.subplots import make_subplots
fig = go.Figure(data=[go.Histogram(x=lbp_image.ravel(), histnorm='probability')])
fig.show()
4. The histogram is now a feature descriptor representing the window at hand.¶
prob b)¶
Come up with a descriptor to represent the whole image as consisting of multiple windows.
Hint: Think of combining several local descriptions into a global description.
First, set the number of bins in which you want to devide the image.
Then devide the image.
#Set the bin here
bins = 5
width_window = lbp_image.shape[1] // bins
height_window = lbp_image.shape[0] // bins
windows=[]
for j in range (0, height_window*bins, height_window):
for i in range(0, width_window*bins, width_window):
windows.append(lbp_image[j:j+height_window,i:i+width_window])
#plot patches
plt.figure(figsize=(15,10))
for i, window in enumerate(windows):
plt.subplot(bins, bins, i+1)
plt.axis('off')
plt.imshow(window, cmap='gray')
plt.show()
We may get BLP codes for each patch.
fig = make_subplots(rows=bins, cols=bins, start_cell="top-left")
for i, window in enumerate(windows):
row, col=i//bins+1, i%bins+1
fig.add_trace(go.Histogram(x=window.ravel(), histnorm='probability'), row, col)
fig.update_xaxes(showticklabels=False)
fig.update_yaxes(showticklabels=False)
fig.update_layout(template="plotly_white",showlegend=False)
fig.show()
prob c)¶
Using the global descriptor you have created, implement a classification process to classify the images in the provided dataset into two categories: face images and non-face images.
Comment the results. Hint: You can use simple histogram similarities.
Is the global descriptor able/unable to represent whole images of different types, e.g. faces vs. cars?
Identify problems (if any).
Suggest solutions if possible.
The dataset consists of a bunch of images of people’s faces taken from MIT Faces Recognition Project database.
import glob
files = glob.glob('facedata/*.pgm')
people = [np.asarray(Image.open(file).convert('L')) for file in files]
files = glob.glob('cardata/*.jpg')
cars = [np.asarray(Image.open(file).convert('L').resize((64,64))) for file in files]
plot([people[0],cars[0]])
people_lbp = np.zeros((20,64,64))
cars_lbp = np.zeros((20,64,64))
for i in range(20):
for x in range(1,63):
for y in range(1,63):
people_lbp[i,y,x] = lbp(people[i], x, y)
cars_lbp[i,y,x] = lbp(cars[i], x, y)
plot([people_lbp[0],cars_lbp[0]])
facehist=np.asarray([np.histogram(people_lbp[i,:,:], bins=255)[0] for i in range(20)])
carhist=np.asarray([np.histogram(cars_lbp[i,:,:], bins=255)[0] for i in range(20)])
faceavg = np.average(facehist, axis=0)
caravg = np.average(carhist, axis=0)
threshold = (faceavg+caravg)/2
print(threshold)
facecorrect = 0
for i in range(20):
if np.average(facehist[i]-threshold[i]) >= 12:
facecorrect += 1
carcorrect = 0
for i in range(20):
if np.average(carhist[i]-threshold[i]) < 12:
carcorrect += 1
print(facecorrect, carcorrect)
prob d)¶
Decrease the window size and perform classification again. Comment the results.
prob e)¶
Increase the window size and perform classification again. Comment the results.
prob f)¶
Discuss how LBP can be used/modified for dynamic texture analysis, e.g. in a video.